Vision and analog sensing scrap sorting system and method

ABSTRACT

A system and a method of sorting scrap particles is provided. A moving conveyor containing scrap particles is imaged using a vision system to create a vision image corresponding to a timed location of the conveyor, and is sensed using a sensing system to create a sensing matrix corresponding to the timed location. The sensing system has at least one array of analog proximity sensors. A control system analyzes the vision image as a vision matrix of cells, and generates a vision vector containing vision data from the vision matrix for the particle. The control system analyzes the sensing matrix, and generates a sensing data vector containing sensing data from the sensing matrix for the particle. The control system classifies the particle into one of at least two classifications of a material as a function of the vision data vector and the sensing data vector.

TECHNICAL FIELD

Various embodiments relate to a system and method for sorting scrapmaterials, including scrap materials containing metal, in a lineoperation.

BACKGROUND

Scrap metals are currently sorted at high speed or high volume using aconveyor belt or other line operations using a variety of techniquesincluding: hand sorting by a line operator, air sorting, vibratorysorting, magnetic sorting, spectroscopic sorting, and the like. Thescrap materials are typically shredded before sorting and requiresorting to facilitate separation and reuse of materials in the scrap,for example, by sorting based on classification or type of material. Bysorting, the scrap materials may be reused instead of going to alandfill or incinerator. Additionally, use of sorted scrap materialutilizes less energy and is more environmentally beneficial incomparison to refining virgin feedstock from ore or manufacturingplastic from oil. Sorted scrap materials may be used in place of virginfeedstock by manufacturers if the quality of the sorted material meets aspecified standard. The scrap materials may be classified as metals,plastics, and the like, and may also be further classified into types ofmetals, types of plastics, etc. For example, it may be desirable toclassify and sort the scrap material into types of ferrous andnon-ferrous metals, heavy metals, high value metals such as copper,nickel or titanium, cast or wrought metals, and other various alloys.

SUMMARY

In an embodiment, a method of sorting scrap particles is provided. Amoving conveyor containing scrap particles is imaged using a visionsystem to create a vision image corresponding to a timed location of theconveyor. A control system is employed to analyze the vision image as avision matrix of cells, identify cells in the vision matrix containing aparticle, and generate a vision vector containing vision data from thevision matrix for the particle. The moving conveyor containing scrapparticles is sensed using a sensing system to create a sensing matrixcorresponding to the timed location of the conveyor, with the sensingsystem having at least one array of analog proximity sensors. Thecontrol system is employed to analyze the sensing matrix, identify cellsin the sensing matrix containing a particle, and generate a sensing datavector containing sensing data from the sensing matrix for the particle.The control system is employed to classify the particle into one of atleast two classifications of a material as a function of the vision datavector and the sensing data vector.

In another embodiment, a system for sorting randomly positioned scrapmaterial particles on a moving conveyor is provided. The system has avision system with an imaging sensor and an illuminated predefinedviewing area to image a conveyor passing therethrough at a timeinterval. The system has a sensor system with an array of analoginductive proximity sensors arranged in a single common plane arrangedgenerally parallel to the conveyor. A control system is configured toreceive and process image data acquired from the vision system toidentify a scrap particle on the conveyor in the viewing area, analyzethe vision data for the particle to form a vision data vector, receiveand process sensor data acquired from the sensing system and timed tocorrespond with the vision data to identify the scrap particle on theconveyor, analyze the sensor data for the particle to form a sensor datavector, and classify the particle into a classification of materialusing the vision data vector and the sensing data vector.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a side schematic view of a sorting system accordingto an embodiment;

FIG. 2 illustrates a top schematic view of the sorting system of FIG. 1;

FIG. 3 illustrates a top view of a sensor assembly for use with thesorting system of FIG. 1 according to an embodiment;

FIG. 4 illustrates a schematic of a sensor interacting with a scrapparticle;

FIG. 5 illustrates a flow chart illustrating a method for classifyingscrap material using the system of FIG. 1;

FIG. 6 illustrates a flow chart for a classification method for use withthe method of FIG. 5;

FIGS. 7A and 7B illustrate vision and sensor images for cast aluminummaterials for use by the method of FIGS. 5 and 6 and obtained using thesystem of FIGS. 1-2;

FIGS. 8A and 8B illustrate vision and sensor images for wrought aluminummaterials for use by the method of FIGS. 5 and 6 and obtained using thesystem of FIGS. 1-2;

FIG. 9 is a plot of sample data for use in setting calibration andclassification parameters;

FIG. 10 illustrates a flow chart for another classification method foruse with the method of FIG. 5; and

FIG. 11 illustrates zoning of the principal component space for thecreation of a look-up table for use with the classification method ofFIG. 10.

DETAILED DESCRIPTION

As required, detailed embodiments of the present disclosure are providedherein; however, it is to be understood that the disclosed embodimentsare examples and may be embodied in various and alternative forms. Thefigures are not necessarily to scale; some features may be exaggeratedor minimized to show details of particular components. Therefore,specific structural and functional details disclosed herein are not tobe interpreted as limiting, but merely as a representative basis forteaching one skilled in the art to variously employ the presentdisclosure.

It is recognized that any circuit or other electrical device disclosedherein may include any number of microprocessors, integrated circuits,memory devices (e.g., FLASH, random access memory (RAM), read onlymemory (ROM), electrically programmable read only memory (EPROM),electrically erasable programmable read only memory (EEPROM), or othersuitable variants thereof) and software which co-act with one another toperform operation(s) disclosed herein. In addition, any one or more ofthe electrical devices as disclosed herein may be configured to executea computer-program that is embodied in a non-transitory computerreadable medium that is programmed to perform any number of thefunctions as disclosed herein.

FIGS. 1-2 illustrate a system 100 or apparatus for classifying scrapmaterials into two or more classifications of materials, and thensorting the materials into their assigned classification. In otherexamples, the system 100 may be used or integrated with otherclassification and sorting systems, for example, in a larger lineoperation for classifying and sorting scrap materials.

A conveyor belt 102, or other mechanism for moving objects along a pathor in a direction, shown here as the y-direction, supports particles 104to be sorted. The particles 104 to be sorted are made up of pieces ofscrap materials, such as scrap materials from a vehicle, airplane,consumer electronics, a recycling center; or other solid scrap materialsas are known in the art. The materials 104 are typically broken up intosmaller pieces on the order of centimeters or millimeters by a shreddingprocess, or the like, before going through the sorting system 100 or alarger sorting facility. The particles 104 may be randomly positionedand oriented on the conveyor 102 in a single layer, have random andwidely varying shapes, and have varying properties. The particles 104may include mixed materials. In one example, the scrap material includeswire, and a particle 104 may include wire in various shapes, includingthree-dimensional shapes. In another example, the particles 104 mayinclude a mixture of cast and wrought materials, such as cast aluminumalloy and wrought aluminum alloy.

The system 100 classifies and sorts the particles 104 into two or moreselected categories of materials. In one example, a binary sort isperformed to sort the materials 104 into two categories. In anotherexample, the materials are sorted into three or more categories ofmaterials. The conveyor belt 102 extends width-wise and transversely inthe x-direction, and pieces or particles of material 104 are positionedat random on the belt 102. In various examples, different scrapmaterials may be sorted, e.g. metal versus non-metal, types of mixedmetals, cast versus wrought, wire versus non-wire, etc.

At least some of the scrap particles 104 may include stainless steel,steel, aluminum, titanium, copper, precious metals including gold, andother metals and metal alloys. The scrap particles 104 may additionallycontain certain metal oxides with sufficient electrical conductivity forsensing and sorting. Additionally, the scrap particles 104 may be mixedmaterials such as metal wire that is coated with a layer of insulation,portions of circuit boards and other electronic waste, scrap materialsfrom tire with metal belt embedded in rubber, and other metals that areat least partially entrapped, encapsulated, or embedded withininsulation, rubber, plastics, or other nonconductive materials. Thescrap materials 104 may be provided as nonferrous materials that containother metal and metal alloys. Note that conductive as referred to withinthis disclosure means that the particle is electrically conductive, orcontains metal. Nonconductive as referred to herein means electricallynonconductive, and generally includes plastics, rubber, paper, and othermaterials having a resistivity greater than approximately one mOhm·cm.

A scrap particle 104 provided by wire may be difficult to detect usingother conventional classification and sorting techniques, as ittypically has a low mass with a stringy or other convoluted shape andmay be coated, which generally provides a barely discernable signal.Alternatively, the scrap particle 104 may be difficult to sort usingconventional classification and sorting techniques for similar materialsthat have been processed using different techniques, such as cast versuswrought materials. The system 100 according to the present disclosure isable to sense and sort these categories of scrap material.

The scrap materials 104 may be shredded or otherwise processed beforeuse with the system 100. Additionally, the scrap materials 104 may besized, for example, using an air knife or another sizing system prior touse with the system 100. In one example, the scrap particles may berough sorted prior to use with the system 100, for example, using asystem containing digital inductive proximity sensors to classify andseparate conductive from nonconductive materials, or using a magneticsorting system to remove ferrous from non-ferrous materials. Theparticles 104 may be per-sorted using an eddy current separator or otherdevice to rough sort the materials prior to being sorted into a desiredclassification by the system 100. Generally, the scrap particles 104 areshredded and sized to have an effective diameter that is similar or onthe same order as a sensor end face diameter. The particles 104 are thendistributed onto the belt 102 as a single layer of dispersed particlesto avoid overlap between particles, and provide separation betweenadjacent particles for both sensing and sorting purposes. The particles104 may be dried prior to distribution, sensing, or sorting to improveefficiency and effectiveness of the sorting process.

The particles 104 of scrap material are provided to a first end region120 of the belt 102. The belt 102 is moved using one or more motors andsupport rollers 122. A control system 112 including a control unit 114controls the motor(s) 122 to control the movement and speed of the belt102.

The belt 102 may pass adjacent to a cleaning system or device 124 shownin FIG. 1. The cleaning system 124 extends across the width of the belt102 and may include a pressurized air source to remove dust, a wettingdevice to provide a spray of liquid, e.g. water, onto the surface of theconveyor 102 to clean the conveyor or create a darkened, more uniformsurface of the conveyor as the background, and/or other devices such asbrushes. The cleaning system 124 may interact with the conveyor 102prior to the particles being deposited onto the belt, and in someexamples, is located along the conveyor return path and beneath theconveyor.

The system 100 has a vision system 106 that images the belt 102 as itpasses through a viewing area of the vision system 106. In one example,the vision system 106 provides a color image in the visible spectrum. Inother examples, the vision system 106 provides another multi-channelimage. The belt 102 passes through the vision system 106, which includesan imaging device 140 to image the material as it moves through thesystem 106. The vision system 106 creates an image of a region of thebelt 102 based on a viewing area of the associated imaging device 140.

The system also has a sensing system 108, or a sensing apparatus 108,that provides sensing data as the belt 102 passes. In one example, andas described below, the sensing apparatus 108 contains one or morearrays of sensors such as analog proximity sensors. In the exampleshown, one sensor array 110 are shown; however, the system 100 may havea single array 110, or more than two arrays 110. Each array 110 includesa plurality of analog proximity sensors, as described in greater detailbelow, and the sensors in the array 110 provide an analog signal inresponse to sensing a particle 104 on the conveyor 102.

The sensors in each array 110 are provided as analog proximity sensors,as opposed to digital sensors. For an analog sensor, the signal outputmay vary and be any value within a range of values, for example, avoltage range. Conversely, with a digital signal, the signal output mayonly be provided as a binary signal, e.g. 0 or 1, or as one of a set ofdiscrete, limited values. The sorting and classification system 100 ofthe present disclosure uses analog sensors to provide greater resolutionin the signal. For example, the analog sensor may output a directcurrent voltage that varies between 0 and 12 Volts, and the signal maybe any value within that range, e.g. 4.23 Volts. For a digital sensor,the signal output may be one of two discrete values, for example, thatcorrespond to voltage values on either side of a set threshold value.

The vision system 106 and sensing system 108 are illustrated as beingarranged sequentially in the system 100 with particles on the belt 102passing the vision system 106 prior to the sensing apparatus. In otherexamples, the vision system 106 may be positioned subsequent to thesensing system 108 along the belt as shown in broken lines in FIG. 1, orthe vision system 106 may be positioned directed above the sensingapparatus 108 and located at the same location along the belt 102.

The motors and support rollers 122 are positioned such that the array110 is directly adjacent to the belt 102 carrying the particles. Forexample, the belt 102 may be directly positioned between the particles104 that it supports and an array 110 such that the array 110 isdirectly underneath a region of the belt 102 carrying particles 104. Themotors and support rollers 122 may direct the returning belt below thearray 110, such that the array 110 is positioned within the closed loopformed by the belt 102.

The vision system 106 and the sensing system 108 provide vision data andsensing data, respectively, to a control system 112 that uses the visionand sensing data to classify the particles as described below, forexample, using a multi-discriminant analysis.

The control system 112 and control unit 114 may include or be incommunication with one or more position sensors 126 to determine alocation and timing of the belt 102 for use locating and trackingparticles 104 as they move through the system on the belt. The positionsensor(s) 126 may be provided by a digital encoder, an optical encoder,or the like. In one example, the conveyor 102 is linearly moved at aspeed on the order of 200 to 800 feet per minute, although other speedsare contemplated. In a further example, the belt 102 has a linear speedof 300-500 feet per minute, and may have a speed of 400 feet per minutecorresponding to a belt movement of 2 millimeters per millisecond, oranother similar speed. The speed may be selected to allow sufficientexposure time to the vision and sensor systems while meeting a desiredthroughput of particles.

The control system 112 uses at least the color data and the sensing dataas described below to identify particles 104 on the belt 102 andclassify each particle 104 into one of a plurality of classifications.The control system 112 then controls a separator unit 128, using theclassification for each particle 104, the location of the particles, andthe conveyor belt 102 position to sort and separate the particles 104.

The system 100 includes the separator unit 128 at a second end 130 ofthe conveyor 102. The separator unit 128 includes a system of ejectors132 used to separate the particles 104 based on their classification.The separator unit 128 may have a separator controller 134 that is incommunication with the control system 112 and the position sensor 126 toselectively activate the appropriate ejectors 132 to separate selectedscrap particles 104 located on the conveyor which have reached thedischarge end 130 of the belt. The ejectors 132 may be used to sort theparticles 104 into two categories, three categories, or any other numberof categories of materials. The ejectors 132 may be pneumatic,mechanical, or other as is known in the art. In one example, theejectors 132 are air nozzles that are selectively activated to direct ajet of air onto selected scrap particles 104 to alter the trajectory ofthe selected particle as it leaves the conveyor belt so that theparticles are selectively directed and sorted into separate bins 136,for example using a splitter box 138.

A recycle loop may also be present in the system 100. If present, therecycle loop takes particles 104 that could not be classified andreroutes them through the system 100 for rescanning and resorting into acategory.

The vision system 106 includes the imaging device 140 and a lightingsystem 142 that the belt 102 and particles 104 pass under. The imagingdevice 140 may be a camera that has a digital color sensor, such as acharge coupled device (CCD) or complimentary-metal-oxide-semiconductor(CMOS) sensor. In one example, the imaging device 140 is a linescancamera that scans with sufficient frequency to provide a continuous feedof sequential image frames of the conveyor 102, and is a three chip, RGBcolor CCD camera. In other examples, the imaging device 140 may includea CMOS sensor or another sensor, or may provide an image with usinganother color model, such as HSV and HSL or other channels. The camera140 has an associated viewing area that is focused on the belt 102. Thecamera 140 may be a multispectral or hyperspectral camera providingultraviolet, visible, and/or infrared channels.

The vision system 106 may additionally or alternatively include athree-dimensional (3-D) imaging device 144 with its own lighting system146. The 3-D imaging device 144 may be a camera or pair of cameras thatviews a laser line profile generated by a line laser 148 and uses thevertical displacement of the line laser to determine the top surfaceprofile of the particle 104. In another embodiment, it may be atime-of-flight laser ranging system coupled to a rapid one-dimensional(1-D) scanning mirror that scans the width of the belt 102.Alternatively, a stereo/dual camera 3D system or any other 3D imagingsystem may be used as is known in the art.

The lighting system 142 illuminates the viewing area of the belt 102 toprovide a controlled uniform illumination of the viewing area for theimaging device 140. The lighting system 142 may be provided with ashroud 150 that includes a frame that supports one or more lightsemitting broadband visible light, such as fluorescent light bulbs,broadband LEDs or halogen light bulbs. The lighting system 142 may alsoinclude a cylindrical lens for collimation and uniform illuminationand/or one or more diffuser panels positioned between the lights and theconveyor 102.

The control system 112 controls the vision system 106 using informationregarding the position of the conveyor 102, for example, using inputsfrom the position sensor 126, to determine the linear advancement of theconveyor belt 102 and the associated advancement of the scrap particles104 on the belt. The control system 112 may control the vision system106 to acquire an image of the viewing area when the conveyor belt 102has advanced a distance equal to the length of the viewing area. Theimaging device 140 includes an image detector or sensor thatelectronically records an image of the viewing area through which thescrap particles are conveyed by the conveyor 102.

FIG. 3 illustrates a top view of a sensing assembly 108 according to anembodiment, and FIG. 4 illustrates a schematic of a sensor in thesensing assembly for use with the system 100 of FIGS. 1-3. The sensingsystem 108, or the sensing apparatus 108, provides sensing data as thebelt 102 passes.

In one example, and as described below, the sensing apparatus 108contains one or more arrays 110 of sensors such as analog proximitysensors. In the example shown, one sensor array 110 is shown; however,the system 100 may have more than one array 110. Each array 110 includesa plurality of analog proximity sensors, as described in greater detailbelow, and the sensors in the array 110 provide an analog signal inresponse to sensing a particle 104 on the conveyor 102.

In the present example, the system 100 uses analog inductive proximitysensors, such that the system is used to sort between two or moreclasses of metals, as the sensors can only detect electricallyconductive materials. Additionally, the system 100 may be used to sortscrap material that includes particles 104 with mixed composition, forexample, insulated wire or other coated wire. In various examples, thesystem 100 is used to sort between at least two of the following groups:metal wire, metal particles, and steel and/or stainless steel, where themetal particles have a conductivity that lies between the wire andsteel/stainless steel groups and may include copper, aluminum, andalloys thereof. The system 100 may be used to sort scrap particles 104having an effective diameter as large as 25 centimeters or more, and assmall as 2 millimeters or 22-24 gauge wire. In other examples, thesystem 100 may be used to sort scrap particles 104 containing metal fromscrap particles 104 that do not contain metal.

The sensor array is arranged in a base member 160 that extendstransversely across the conveyor belt 102. The base member 160 providessupport for and positions an array of sensors. In one example, the basemember 160 is provided by a sensor plate that defines an array ofapertures 162 that intersect the upper surface, with each aperture sizedto receive a corresponding sensor 170 in the array 110 of analogproximity sensors. In other embodiments, other structure or supports maybe used to position and fix the sensors into the array in the assembly.The base member 160 provides for cable routing for a power harness 164to provide electrical power to each of the sensors 170 and also for adata harness 166 to transmit analog signals from each of the sensors 170to a signal processing unit 190, or sensor processor 190, in the controlsystem 112.

Each sensor has an end surface or active sensing surface 172. Thesensors 170 are arranged into an array 110 such that the end surfaces172 of each of the sensors are co-planar with one another, and lie in aplane that is parallel with the surface 116 of the belt, or generallyparallel to the surface of the belt, e.g. within five degrees of oneanother, or within a reasonable margin of error or tolerance. The endfaces 162 of the sensors likewise generally lie in a common plane, e.g.within an acceptable margin of error or tolerance, such as within 5-10%of a sensor end face diameter of one another or less. The sensors 170are arranged in a series of rows 168 in the array 110, with each row 168positioned to extend transversely across the sensor assembly 108 andacross a belt 102 when the sensor assembly is used with the system 100.Each row 168 in the array 110 may have the same number of sensors 170 asshown, or may have a different number. The sensors 170 in one row 168are offset from sensors 170 in an adjacent row 168 along a transversedirection as shown to provide sensing coverage of the width of the belt102. The sensors 170 in the array 110 are arranged such that, in theX-position or transverse direction and ignoring the Y-position, adjacentsensors have overlapping or adjacent electromagnetic fields. The sensors170 may be spaced to reduce interference or crosstalk between adjacentsensors in the same row 168, and between sensors in adjacent rows 168.In one example, all of the sensors 170 in the array are the same typeand size of sensor. In other examples, the sensors 170 in the array maybe different sizes, for example, two, three, or more different sizes.

The sensors 170 may be selected based on the size of the active sensingarea, or a surface area of the end face 172. The sensors are alsoselected based on their sensitivity and response rate. In one example,the end face 172 area generally corresponds with or is on the same orderas the size of the particles 104 to be sorted, for example, such thatthe sensor is used to sort particles having a projected area within 50%,20%, or 10% of the sensor surface area. For example, the sensor endsurface 172 area may be in the range of 2 millimeters to 25 millimeters,and in one example is on the order of 12-15 or 15-20 millimeters for usewith scrap particles 104 having an effective diameter in the same sizerange, e.g. within a factor of two or more. Therefore, although thescrap materials 104 may undergo a rough sorting process prior to beingdistributed onto the belt, the system 100 allows for size variation inthe scrap particles. In another example, the end face 172 area may beselected to be smaller than the size of the particles to be sorted, forexample, such that the sensor is used to sort particles having aprojected area within 200-500% of the sensor surface area. In otherexamples, the sensor end face area and the size of the particles to besorted may have another targeted relationship.

The sensors 170 may be selected based on the materials to be sorted. Inthe present example, the sensors 170 in the array 110 are each inductiveanalog proximity sensors, for example, for use in detecting and sortingmetals. The sensor 170 creates an induction loop as electric current inthe sensor generates a magnetic field. The sensor outputs a signalindicative of the voltage flowing in the loop, which changes based onthe presence of material 104 in the loop and may also change based onthe type or size of metal particles, or for wire versus solid particles.The control system 112 may use the amplitude of the analog voltagesignal to classify the material. In further examples, the control system112 may additionally or alternatively use the rate of change of theanalog voltage signal to classify the material. The control system 112may use at least one of the following as determined from the system 108to classify a region associated with the particle: a peak voltage, arate of change of voltage, an average voltage, a sum of the voltagesover the area associated with the particle region, an area ratio factoras determined using a particle area divided by a bounding box area, acompactness factor as determined as a function of the particle perimeterand the particle area, and the like.

In the present example, the array 110 includes five rows 168 of sensors170, with each row having 24 identical analog inductive proximitysensors, with each sensor having an end face diameter of 18 millimeters.The array 110 therefore contains 120 sensors. The sensors 170 in eachrow 168 are spaced apart from one another by approximately five timesthe diameter of the sensor to reduce crosstalk and interference betweenthe sensors, and in further examples the sensors 170 are spaced apart bymore than five times the diameter of the sensor. The number of sensors170 in each row is therefore a function of the diameter of the sensorand the length of the row which corresponds to the width of the belt.The number of rows 168 is a function of the width of the belt, thenumber and size of sensors, and the desired sensing resolution in thesystem 100. In other examples, the rows may have a greater or fewernumber of sensors, and the array may have a greater or fewer number ofrows, for example, 10 rows.

In the present example, each row 168 is likewise spaced from an adjacentrow by a similar spacing of approximately five times the diameter of thesensor 170, and in further examples the sensors 170 are spaced apart bymore than five times the diameter of the sensor. The sensors 170 in onerow 168 are offset transversely from the sensors in adjacent rows. Thesensors 170 in the array as described provide for a sensor positionedevery 12.5 mm transversely across the belt when the sensor 170 positionsare projected to a common transverse axis, or x-axis, although thesensors 170 may be at different longitudinal locations in the system100. The control unit therefore uses a matrix or linescan image with 120cells in a row to correspond with the sensor arrangement in the array. Ascrap particle 104 positioned at random on the belt is likely to travelover and interact with an electromagnetic field of at least two sensors170 in array. Each sensor 170 has at least one corresponding valve orejector 132 in the blow bar of the sorting assembly.

The end faces 172 of the sensors in the array lie in a single commonplane, or a sensor plane. This plane is parallel to and spaced apartfrom a plane containing the upper surface 116 of the belt, or a beltplane. The sensor plane is spaced apart from the belt plane by adistance D, for example, less than 5 millimeters, less than 2millimeters, or one millimeter. Generally, improved sorting performancemay be provided by reducing D. The distance D that the sensor plane isspaced apart from the belt plane may be the thickness of the belt 102with an additional clearance distance to provide for movement of thebelt 102 over the sensor array 110.

The sensors 170 in the array 110 may all be operated at the samefrequency, such that a measurement of the direct current, analog,voltage amplitude value is used to classify the materials. In otherexamples, additional information from the sensor 170 may be used, forexample, the rate of change of the voltage. As a scrap particle 104moves along the conveyor belt 102, the particle traverses across thearray 110 of sensors. The particle 104 may cross or traverse anelectromagnetic field of one or more of the sensors 170 in the array. Asthe particle 104 enters a sensor electromagnetic field, theelectromagnetic field is disturbed. The voltage measured by the sensor170 changes based on the material or conductivity of the particle, andadditionally may change based on the type or mass of material, e.g. wireversus non-wire. As the sensor 170 is an analog sensor, it provides ananalog signal with data indicative of a material that the sensor issensing, e.g. the amplitude of the direct current voltage measured bythe sensor 170, that may be used to classify the particle.

As the particles 104 are all supported by and resting on the conveyorbelt 102, the scrap particles all rest on a common belt plane that iscoplanar with the sensor plane of the sensor array 110. As such, thebottom surface of each particle is equidistant from the sensor array asit passes overhead by the distance D. The scrap particles in the system100 have a similar size, as provided by a sizing and sorting process;however, there may be differences in the sizes of the scrap particles,as well as in the shapes of the particles such that the upper surface ofthe particles on the belt may be different distances above the sensorarray. The particles therefore may have a thickness, or distance betweenthe bottom surface in contact with the belt and the opposite uppersurface that is different between different particles being sorted bythe system 100. The scrap particles interact with the sensors in thearray to a certain thickness, which corresponds with a penetration depthof the sensor as determined by the sensor size and current.

FIG. 4 illustrates a partial schematic cross-sectional view of a sensor170 in an array 110 and a particle 104 on a belt 102. As can be seenfrom the Figure, the upper surface 116 of the belt 102, or belt plane,is a distance D above a sensor plane containing the end face 172 of thesensor 170. The sensor 170 contains an inductive coil 174 made fromturns of wire such as copper and an electronics module 176 that containsan electronic oscillator and a capacitor. The sensor 170 receives powerfrom an outside power supply. The inductive coil 174 and the capacitorof the electronics module 176 produce a sine wave oscillation at afrequency that is sustained via the power supply. An electromagneticfield is produced by the oscillation and extends out from the end face172, or the active surface 172 of the sensor 170. An electromagneticfield that is undisturbed by a conductive particle, e.g. when there isno scrap material on the belt 102, is shown at 178. When a scrapparticle 104 containing a conductive material, such as metal, enters theelectromagnetic field, some of the oscillation energy transfers into thescrap particle 104 and creates eddy currents. The scrap particle andeddy current result in a power loss or reduction in the sensor 170, andthe resulting electromagnetic field 180 has a reduced amplitude. Theamplitude, e.g. the voltage, of the sensor 170 is provided as a signalout of the sensor via the output. Note that for an analog sensor, thesensor 170 may continually provide an output signal, for example, as avariable voltage within a range of voltages, that is periodicallysampled or acquired by the control system 112.

Referring back to FIGS. 1-2, the control system 112 receives the imagesfrom the vision system 106, and uses the images to locate and trackparticles 104 of material on the belt. The control system 112 uses theimaging data to determine vision data or color data associated with eachparticle 104 on the belt 102. The control system 112 also uses signalsfrom the sensing apparatus 108 to determine sensing data associated witheach particle 104. The control system 112 may use the vision and/or thesensing data to locate and track particles 104 on the belt 102. Thecontrol system 112 uses the vision data and the sensing data for use insorting the particles 104 into two or more classifications as theparticles move along the belt. In one example, the control system 112conducts a multi-discriminant analysis combining vision and sensing datafor the particle to classify the particle.

The control system 112 and control unit 114 may be provided by anetworked computer system employing a plurality of processors to achievea high-speed, multi-tasking environment in which processing takes placecontinuously and simultaneously on a number of different processors. Inthe control system 112, each processor in turn is capable of providing amulti-tasking environment where a number of functionally differentprograms could be simultaneously active, sharing the processor on apriority and need basis. The choice of implementation of hardware tosupport the functions identified in the process groups may also dependupon the size and speed of the system, as well as upon the categoriesbeing sorted.

An image processing unit 192, or image processor 192, is provided in thecontrol unit to periodically acquire and process the images. The imageprocessor 192 includes logic for assembling each image from the camera.The image processor 192 may divide each image of the viewing area into amatrix of cells, and analyze the digital data corresponding to the imageto determine locations of particles 104 on the conveyor 102 for use inthe separator unit 128, and to determine or extract vision data for eachparticle 104 for use in the classification and sorting process. Theimage processor 192 receives signal indicative of the position of theconveyor 102 and when to acquire an image such that the conveyor belt isimaged in a series of sequential images of discretized sections of theconveyor as it passes through the viewing area. The control system 114and image processor 192 of the control unit 112 may perform variousanalyses on each of the digital pixel values recorded for an image asdescribed below.

The control system 112 may include a signal processing unit 190, orsensor processor 190, for example to quantize and digitize the signalsfrom the array 110 for use by control unit 114 in classifying andsorting the particles 104. The sensor processor 190 may quantize anddigitize the analog signal to maintain a predetermined resolution in thesignal and data, for example, to tenths or hundredths of a volt, or mayconvert the analog signal to an 8-bit (or higher precision) value.

The control system 112 controls the sensing assembly 108 usinginformation regarding the position of the conveyor 102, for example,using inputs from the position sensor 126, to determine the linearadvancement of the conveyor belt 102 and the associated advancement ofthe scrap particles 104 on the belt. The control system 112 may controlthe sensor processor 190 and sensing assembly 108 to acquire sensor datawhen the conveyor belt 102 has advanced a predetermined distance and tocorrespond with vision data for a location on the belt 102 such that thesame particles are being detected and analyzed by both systems 106, 108.

The control system 112 contains one or more data processing units 190,192 to acquire and process the signals and data from the systems 106,108. In one example, the data processing units 190, 192 are integratedwith the control unit 114, and in other embodiments, the processingunits are separate.

The sensor processor 190 for the sensor system 108 includes logic forassembling the data from each sensor into a representation of the belt102. The sensor processor 190 may represent a transverse section of thebelt as a matrix of cells, and analyze the sensor data to determinelocations of particles 104 on the conveyor 102, and to determine aninput for each particle 104 for use in the classification and sortingprocess. The sensor processor 190 receives a signal indicative of theposition of the conveyor 102 and when to acquire sensor data such thatthe conveyor belt is “imaged” in a series of discretized sections of theconveyor 102 as it passes across the sensor assembly 108 and array 110and creates a sensor data matrix that is a linescan image of the belt.The control system 112 and sensor processor 190 may perform variousanalyses on the sensor data matrix as described below, or otherwisemanipulate the sensor data.

The control system 112 uses vision data and the sensing data, e.g. thequantized and digitized signals from the sensing assembly 108, toclassify the particle 104 into one of two or more preselectedclassifications. Based on the classification outcome, the control system112 controls the separator unit 128 to sort the particles 104 based ontheir associated classifications. The control system 112 may alsoinclude one or more display screens and a human machine interface 194,for use in controlling the system 100 during operation and also for usein calibration or system setup.

FIG. 5 illustrates a method 200 for classifying particles 104 using thecontrol system 112 of the system 100 as shown in FIGS. 1-4. In otherembodiments, various steps in the method 200 may be combined,rearranged, or omitted.

At step 202, the control system 112 provides a line trigger signal tothe camera 140 to acquire a single line based on the position of theconveyor 102. In response to receiving the line trigger signal, thecamera 140 acquires a line scan image. The camera may be controlled toacquire multiple consecutive lines as the belt moves to create an imageor frame of a region of the belt.

At 204, the control system 112 forms a first matrix, or image matrixassociated with the line scan image that is also linked to the positionor coordinates of the belt 102 for use by the separator unit 128 and forcoordination with the sensing apparatus 108 and sensing dataacquisition. At step 202, the image matrix overlays the image such thateach cell in the matrix is associated with one or more pixels in theimage. In one example, the image matrix may have a cell associated witheach pixel. In other examples, the image matrix may have one cellassociated with multiple adjacent pixels. The image matrix may be sizedto correspond with the size of the sensor matrix as described below, forexample, with the same aspect ratio.

The image processor 192 or control system 112 may use a matrix withcells and arrays of the matrix including [R, G, B] color channel data,and additional information regarding particle location, and particleproperties as determined below. The image processor 192 or controlsystem 112 may alternatively use an imaging library processing tool,such as MATROX, to create a table or other database populated with pixeldata for each particle including [R, G, B] values, boundary information,and other particle properties as determined below.

Each image matrix may be formed using RGB signals corresponding to eachof the red, green and blue segments or color components of the viewingarea as detected by the sensor in the camera 140. In other examples,other color signals may be provided to the image processor 192 that arebased on a different color space and color model to indicate a color forthe image that includes at least three-color components or otherchannels may be provided from the camera, e.g. grayscale, non-visiblespectra, and the like. The RGB signals from the camera 140 are assembledor converted by the image processor 192 to three arrays for each imagematrix, with each array corresponding to one of the red, green, and blueimages. The image processor 192 may assemble the line scan image data toform a larger, composite image matrix with the arrays. Each array may be1024×2048 digital pixel values (from 0 to 255) for each of the red,green and blue images. The arrays may be overlaid to provide a colorimage with three channels, or RGB channels, for each pixel or for eachcell in the image matrix. The RGB values are provided as a dataset of[R, G, B] values, each value ranging from 0 to 255, to the pixel or cellin the image matrix.

The image processor 192 may transform the image matrix using a fastFourier transform (FFT), reject high frequency noise and/or rejectspecific frequencies in the X- and/or Y-directions representing beltpattern noise, and then apply an inverse FFT to restore the improvedimage matrix. The control system 112 is employed to reduce noise on theimage by transforming the image via a FFT to create a representation ofthe image in a frequency domain, remove at least one specified frequencyfrom the representation, and transforming the representation back to theimage via an inverse FFT.

The control system 112 may additionally normalize each of the colorchannels for the image, e.g. the R image or array, the B image or array,and the G image or array for an RGB color space, to correct any spectralimbalances in the light sources. For example, each pixel in the imagemay have the R value, G value, and/or B value modified using look-up orcalibration tables associated with each color. The calibration orlook-up tables for the color correction of the image may be obtainedbased on a calibration process using a neutral background, a calibrationcard, or the like.

At 206, the control system 112 identifies cells in the image matrix orpixels in the image that may contain a particle 104 by distinguishingthe particle from a background indicative of the conveyor 102. Theparticle 104 is distinguished from the background by applying athreshold on at least one channel of the image or matrix and flagging apixel or cell when a value of the at least one channel exceeds thethreshold to indicate the presence of a particle 104. The threshold usedto distinguish the particle in the vision matrix may be based on thevision system and vision matrix, and/or the sensing system andcorresponding sensor matrix described below.

As a part of the particle identification process, the control system 112may conduct frame-to-frame stitching by stitching the image matrixbetween image matrices taken at the immediately preceding time andimmediately subsequent time. By stitching the frames together, a largercontinuous region of the belt may be analyzed by the control system 112for particle identification and image data for the particles that are onthe central image matrix. For example, three frames may be stitchedtogether, with the image frame at time t1 stitched between an imageframe from the preceding time t0 and an image frame from the next timet2. The images matrices, including the stitched matrices, may betemporarily stored in a data buffer in memory accessible by the controlsystem 112. As particles may extend across more than one image matrix orframe, the use of stitching aids in particle identification andobtaining complete image data for a single particle on the belt.

At step 208, the control system 112 creates a vision data vector withinformation related to the belt position, an image pointer to thelocation and boundary of the identified particle on the image matrix,and any other image or vision data associated with the particle such ascolor inputs or the like. In one example, the control system 112 usesthe stitched matrices to create the vision vector with reference to theimage pointer and the central matrix or frame. The control system 112additionally further analyzes the identified particle region to provideadditional color data and/or image data and create a vision data vectorfor each identified particle region. The control system 112 may furtherprocess the region of the image or matrix associated with a particle 104using various machine vision processing techniques to erode, dilate,fill holes, or otherwise modify or correct the region of the image ormatrix associated with the identified particle 104.

The control system 112 and image processor 192 may calculate a colorinput for the identified particle, for example, using a color modelbased on color components for each pixel in the image or cell in thematrix associated with the particle. The control system 112 may applycolor criteria to the pixels of the image or the cells of the matrixassociated with the particle 104 to determine the color input for theparticle. In other examples, the control system 112 may evaluate groupsof pixels or cells of the image associated with the particle 104. Forexample, the control system 112 may analyze the frequency and/ordistribution of color components in neighboring pixels or cells indetermining an overall color input for the particle 104.

The control system 112 receives the color components from each pixel ofthe image or cell of the matrix associated with the particle, and eachpixel of the image may have three or more color components, for example,the three RGB channels as described above. For each pixel of eachidentified particle 104, the control system 112 therefore obtains thecolor components, e.g. R,G,B; or H,S,V; or other color space bases suchas those from multispectral camera data with more than 3 colorcomponents.

In one example, the control system 112 may average the R values for allpixels associated with the particle, the B values for all pixelsassociated with the particle, and the G values for all pixels associatedwith the particle, resulting in a color dataset for the particle withthree scalar values, as [R_(average), B_(average), G_(average)], for thevision vector. In other examples, the control system 112 may calculateaverage color component values for the particle based on another colormodel or space, such as HSV, etc. In another example, the control system112 may create a histogram for each color component of the particle 104,such that there is an R histogram, a G histogram, and a B histogram foran identified particle region using a RGB color space, or a singlehistogram for the particle 104 with three sets of bins, with each set ofbins associated with a different color component of the color space asseparated into 8-bit, 16-bit, or otherwise sized bins. The controlsystem 112 may normalize the histogram, for example using the pixel areaof the particle. The resulting input to the vision vector is a datasetcontaining a number of scalar values based on the values associated witheach bin. The control system 112 may additionally or alternativelyemploy a discriminant analysis to determine one or more color inputs orvalues for the vision vector.

The control system 112 may additionally calculate or determine othervisual parameters for the particle for inclusion in the vision vectorincluding: texture features, a color component standard deviation, agrayscale volume, an aspect ratio, dimensionless perimeter (perimeterdivided by square root of area) or another visual characteristic of theidentified particle from the image or matrix as a visual feature for theparticle. Texture features may include rank, number of holes created bythresholding the particle or by subtracting one rank image from another,total hole area as a proportion of total area, largest hole area as aproportion of area, and Haralick texture features. The control system112 may assign texture values to the particle by transforming the imagevia a fast Fourier transform (FFT). The average log-scaled magnitude indifferent frequency bands in the FFT magnitude image may be used asdistinguishing texture features.

In a further example, the vision system 106 may incorporatethree-dimensional vision components, for example, via the addition of alaser profiler or the like. The laser profiler adds an additional arrayof vision data to the image matrix. Various image parameters may beextracted from three-dimensional data frame and added to the imagevector, for example, three-dimensional volume, slope, peak height, adifference between maximum and minimum heights, height ranking, and thelike. For example, differences may be determined between cast materialand wrought material using the three-dimensional data, e.g. castmaterial has a flatter profile than wrought material. In furtherexamples, the three-dimensional vision component includes near-infraredvision data, and additional vision data such as scatter may be added aswell as range and intensity.

At steps 220 and 222, the control system 112 acquires sensor data usingsignals obtained from the sensors in the array 110, and processes thesignals to form a second matrix or sensor matrix. The control system 112creates the sensor matrix using signals received by the sensors in thearray 110 that represents the belt 102 in a similar manner to a linescanimage. If the sensors are not arranged in a single line, the times atwhich data is acquired into a “line scan” are appropriately compensatedaccording to each sensor's distance along the Y direction, i.e. thedirection of particle travel or movement of the belt 102. The controlsystem 112 and sensor processor 190 acquire and process the signals fromthe sensors in the array 110 and sensing assembly 108 to create thesensor matrix from a series of the linescan images. The sensor matrix isformed by a series of rows, with each row representing a narrow band ofthe belt that extends the width of the belt 102. Each row is dividedinto a number of cells, and the processing unit enters data from thesensors into the cells such that the matrix is a representation of theconveyor belt 102, e.g. the matrix represents discretized sections orlocations of the conveyor 102 as it passes across the array 110.

As the control system 112 and sensor processor 190 receive the data fromthe sensors 170, the control system 112 and sensor processor 190 form amatrix or linescan image associated with sensor array 110 that is alsolinked to the position or coordinates of the belt 102 for use by theseparator unit 128 as shown at 204. The sensor processor 190 receivesdata from the sensor array 110, with a signal from each sensor 170 inthe array. The sensor processor 190 receives signals from the sensors,and based on the position of the belt 102, for example, as provided by adigital encoder, inputs data from selected sensors into cells in thematrix. The sensor matrix is acquired at a specified time to correspondto the same belt location as the image matrix, and may have a differentsize or resolution based on the number of sensors. In one example, thesensor matrix is 120×144 in size. The sensor matrix provides arepresentation of the belt 102, with each cell in the matrix associatedwith a sensor 170 in the array. In one example, the sensor matrix mayhave a line with a cell associated with each sensor in the array, withthe cells ordered as the sensors are ordered transversely across thebelt when projected to a common transverse axis. Therefore, adjacentcells in a line of the matrix may be associated with sensors 170 indifferent rows in the array.

The control system 112 and sensor processor 190 receive the digitizeddirect current voltage signal or quantized value from the analoginductive sensor 170. In one example, the quantized value may be an8-bit greyscale value ranging between 0-255. The sensor 170 may outputany value between 0-12, 0-11, 0-10 Volts or another range based on thesensor type, and based on the sensor voltage output, the processorassigns a corresponding bit value. In one example, zero Volts isequivalent to a quantized value of zero. In other examples, zero Voltsis equivalent to a quantized value of 255. In other examples, the sensorprocessor 190 may use other quantized values, such as 4-bit, 16-bit,32-bit, may directly use the voltage values, or the like. For example, asensor 170 that is not sensing a conductive scrap particle has a voltageof 10 Volts, and a sensor sensing a metal particle, such as steel orstainless steel, may have a peak sensor voltage of approximately 2.5Volts, although this may vary based on the thickness of the particle 104over the sensor 170, whether the particle is traveling through theentire electromagnetic field of a sensor 170 or only a portion thereof,etc. The voltage values used in the second matrix may be truncated forsimplicity to the tenth or hundredth of a volt. When an 8-bitclassification value is used with the analog sensors, 10 volts may havea quantized value of 0, with zero Volts having a quantized value of 255,and a voltage of 2.5 Volts having an associated quantized value of 191.

The cells in the sensor matrix are populated with a peak voltage asmeasured by the sensor 170 within a time window or at a timestamp. Inother examples, the sensor signal data may be post-processed to reducenoise, for example, by averaging, normalizing, or otherwise processingthe data.

The sensor processor 190 and control system 112 may use a matrix withcells containing additional information regarding particle location, andparticle properties as determined below. The processor and control unit112 may alternatively use an imaging library processing tool, such asMATROX, to create a table or other database populated with signal datafor each particle including quantized 8-bit voltage values, boundaryinformation, and other particle properties as described below withrespect to further embodiments.

At step 224, the control system 112 identifies cells in the secondmatrix that may contain a particle 104 by distinguishing the particlefrom background signals indicative of the conveyor 102. The particle 104may be distinguished from the background when a group of adjacent cellshave a similar value, or values within a range, to indicate the presenceof a particle 104 or when a single cell is sufficiently different fromthe background, for example, by applying one or more thresholds to thesensor matrix. The control system 112 then groups these sensor matrixcells together and identifies them as a “grouping” indicative of aparticle. The threshold used to distinguish a particle in the sensormatrix may be based on the sensor system and sensor matrix, and/or thevision system and corresponding vision matrix described above.

As a part of the particle identification process, the control system 112may conduct frame-to-frame stitching by stitching the sensor matrixbetween sensor matrices taken at the immediately preceding time andimmediately subsequent time, in a manner similar to that described abovewith respect to the images matrices. By stitching the sensor matricestogether, a larger region of the belt may be analyzed by the controlunit for particle identification and sensor data for the particles thatare on the sensor matrix. In one example, the control system 112 usesthe stitched matrices to create a sensor vector with reference to asensor pointer to the central frame to provide information regardingparticle location. The sensor matrices or stitched sensor matrices maybe temporarily stored in a data buffer in memory accessible by thecontrol system 112. As particles may extend across more than one sensormatrix, the use of frame stitching aids in particle identification andobtaining complete sensor data for a single particle on the belt.

At step 226, the control system 112 creates a sensor data vector foreach identified particle region with information related to the beltposition, a sensor pointer to the location and boundary of theidentified particle on the image matrix, and any other sensor dataassociated with the particle such as voltage values or the like. Thecontrol unit may further process the region of the image or matrixassociated with a particle 104 using various machine vision processingtechniques to modify or correct the region of the matrix associated withthe identified particle 104, or to identify parameters or sensing dataassociated with the particle as indicated below.

The sensor data vector contains information related to the beltposition, the sensor matrix pointer, and any sensor data such as a peakvoltage, an average voltage, a summation of the voltages within theregion identified as the particle, a rate of change of voltage for asensor for the particle, an average voltage, a sum of the voltages overthe area associated with the particle region, an area ratio factor asdetermined using a particle area divided by a bounding box area, acompactness factor as determined as a function of the particle perimeterand the particle area, and the like. For example, the control system 112incorporates a peak voltage from a cell associated with the groupinginto the sensor data vector, for example, the highest or lowest cellvoltage or quantized value in the grouping. In other examples, thecontrol system 112 may provide a value to the sensor data vector for theidentified particle region as a sum of all of the values in the particleregion, an average of all of the cells, as an average of the peakvoltages or quantized values from three cells in the particle region, anaverage of the peak voltages or quantized values from three contiguouscells, or the like. In further examples, the control system 112 mayinput calculated values for the particle into the sensor vector such asshape, size, aspect ratio, texture feature, voltage standard deviation,or another characteristic of the grouping or identified particle fromthe sensor data in the matrix as a secondary feature for the particle.Some secondary classification features, such as texture, may only beobtained with the use of sensors that are smaller than the particlesizing to provide increased resolution and the data required for thistype of analysis.

Although the method 200 is described as having separate particleidentification steps and matrix processing at 204, 206, 222, and 224,variations of the method 200 are also envisaged, as indicated by block228. In one example, the method 200 identifies particles solely usingthe image matrix, and uses the image pointer to create a sensor pointerto a corresponding region in the sensor matrix, regardless of whetherthe sensors 170 have detected any particles in that region. In anotherexample, the method 200 identifies particles solely using the sensormatrix, and uses the sensor pointer to create an image pointer to acorresponding region in the image matrix, regardless of whether thevision system 106 has detected any particles in that region. In afurther example, the method 200 identifies particles using either thesensor matrix or the image matrix, such when a particle is identified ineither the image matrix or the sensor matrix, an image pointer and asensor pointer are created for that region in both matrices. In afurther example, the method 200 identifies particles using both thesensor matrix and the image matrix, such an image pointer and a sensorpointer is created only when a particle is positively identified in boththe image matrix or the sensor matrix.

At step 230, the control system 112 classifies the identified particleregion using the data from both the vision and sensor vectors. Thecontrol system 112 may use a discriminant analysis technique to classifythe particles. In one example, the control system 112 classifies theparticle using a method as illustrated in FIG. 6 or FIG. 10, and asdescribed below in greater detail.

In other examples, the control system 112 may classify the particle byinputting the vectors into a machine learning algorithm. The controlunit may use a Support Vector Machine (SVM), a Partial Least SquaresDiscriminant Analysis (PLSDA), a neural network, a random forest ofdecision trees, a vector space model such as a bag-of-words model, oranother machine learning and classification technique to evaluate thedata vector and classify the particle 104. The control system 112 mayalternatively use independent decisions from the vision data and sensordata, or vision and sensor vectors with a probabilistic combination ofthe two to determine a final classification. In one example, a neuralnetwork is used to classify each of the scrap particles 104 as one of apreselected list of materials based on the analysis of the image andsensor vectors. In other examples, the control system 112 may use alook-up table that plots the data vectors and then classifies thegrouping based on one or more regions, thresholds, or cutoff planes. Inone example, the classification of a particle 104 may be a multiplestage classification. Known algorithmic techniques such as backpropagation and competitive learning, may also be applied to estimatethe various parameters or weights for a given class of input and outputdata.

At step 232, the control system 112 controls the separator unit 128 toselectively activate an ejector 132 to eject a particle into a desiredbin based on the classification for the particle. The control system 112controls the ejectors 132 based on the classification of the particle104, the position of the particle on the belt, and the position andtiming of the conveyor 102.

According to an example, the system 100 is configured to sort particles104 that are small in size and contain metals with higher value, such ascopper, gold, or platinum, that may be mixed with other metal particlesor provided as a coating. The particles 104 may have a largest dimensionon the order of a centimeter or less. The particles 104 may be wettedprior to passing through the vision and sensing systems 106, 108, or maybe provided as dry particles through the system 100. In one example, thecontrol unit 112 may identify a particle 104 location on the belt 102using information from both the sensor and image data.

In other examples, the sensor data is used to identify the location of aparticle on the belt 102, as the vision image may have higher backgroundnoise. In a further example the sensing system 108 is positioned priorto the image system 106 such that the particle location as determinedfrom the sensing system 108 may be used to determine the regions of theimage data for analysis by the control unit 112 and image processingunit and reduce overall processing time. The control unit 112 classifiesthe particles 104 using the vision vector and the sensor vector. Forexample, the control unit may use the vision vector to aid inidentification of gold or copper using color components such as [R, G,B] information in addition to the sensor data. In other examples, thecontrol unit 112 may further use the sensor vector to aid inidentification of light and heavy metals, for example, to classifyparticles between Aluminum or Titanium material.

FIG. 6 illustrates a flow chart for a method 250 of classifying aparticle 104 according to an embodiment and for use by the controlsystem 112 during step 230 of method 200 using the vision and sensorvectors as inputs, and FIGS. 7-9 provide examples of data and aclassification step as used by the method 250. In other embodiments,various steps in the method 250 may be combined, rearranged, or omitted.With reference to the method 250, an example using cast and wroughtaluminum and aluminum alloy materials is used by way of a non-limitingexample, and one of ordinary skill in the art would understand how toapply the method 250 to classifying other groupings of materials. Thevision vector and sensor vector are used as inputs to a discriminantanalysis conducted by the control system 112 implementing method 250 toclassify the scrap particles 104 as being cast aluminum or wroughtaluminum.

FIG. 7A illustrates an image from the vision system 106 of fourdifferent particles formed from cast aluminum. FIG. 7B illustrates datafrom the sensing system 108 for the same four cast aluminum particles,and in the same order left-to-right.

FIG. 8A illustrates an image from the vision system 106 of fourdifferent particles formed from wrought aluminum. FIG. 7B illustratesdata from the sensing system 108 for the same four wrought aluminumparticles, and in the same order left-to-right.

Referring back to FIG. 6, at step 252, the control system 112 inputs thevision and sensor vectors into the classification method 250, forexample, after analyzing one of the particles as shown in FIG. 7 or 8.The vision vector and the sensor vector according to the present examplehave a total of twelve parameters or scalar values, although othernumbers of parameters are also contemplated in other embodiments. Thevision vector for each particle may include: the vision pointer, redchannel average, green channel average, blue channel average, a busynessscore, a perimeter value, and at least one hole determination for eachidentified particle region, and other vision values. The busyness scoreis calculated by comparing a pixel or cell to its adjacent pixels orcells and determining a brightness and variation in intensity, with thebusyness score being a normalized sum for the particle region.Generally, wrought material has a higher busyness score than castmaterial. The perimeter value may provide an indication of thecompactness of the identified particle region and particle, and becalculated using the perimeter of the particle divided by an areafactor, such as the square root of the particle area. The “hole”determination may be based on saturated pixels in the image and includeat least one of the following from the matrix, image, or an enhancedcontrast image: a value representing the number of holes or saturatedpixels measured for the identified particle region, an area of thelargest hole in the identified particle region normalized by the area ofthe identified particle, and a sum of the any hole areas in theidentified particle region normalized by the area of the identifiedparticle. Note that wrought materials tend to have more saturatedpixels, or “holes”, than cast material. Other vision values may includean area ratio to provide an indication of the compactness of theparticle, e.g. the ratio of the actual area of the identified particleregion normalized by an area of a bounding box about the same particleregion. Note that cast materials tend to have a larger area ratio thanwrought materials as they are more compact.

The sensor data vector for each particle may include the sensor pointer,an average voltage, a peak voltage, and an area ratio factor. In otherexamples, the sensor data vector may include additional values asdescribed above. Note that cast materials tend to have a lower voltageoutput from the system 108, leading to a brighter appearance in thesensor image.

Additional values or inputs for the discriminant analysis may beprovided by combining the sensor and vision vectors, with the controlsystem 112 conducting further calculations and analyses based on thedata. For example, the control system 112 may calculate an area ratiobetween the areas of the particle from the vision data and sensor data,respectively, and associate this with one of the sensor or visionvectors or in a combined vector with all of the data and parameters.

At step 254, an initial classification may be performed by the controlsystem 112 to classify particles that easily are identified as being inone of the categories. For example, any non-metal materials may beeasily identified in comparison to metal materials based on a comparisonof the vision and sensing data. As non-metals lack conductivity, thevision vector would indicate a particle while the sensor vector wouldnot indicate a particle. Additionally, particles with embedded metalsmay be easily identified and classified based on a comparison of visionand sensing data. Furthermore, some particles may easily be classifiedat this step by the control system 112 as being either cast or wroughtbased on the vision and/or sensor data and vectors. For example, aparticle with a conductivity below a threshold value may be identifiedand classified as being cast material using the sensor vector, while anyparticles with a size or area greater than a specified threshold may beidentified and classified as being wrought material using either thesensor or vision vectors. This step may also be used as a pre-filterusing a pre-determined parameter such as aspect ratio or elongation.

At step 256, the control system 112 conducts a discriminant analysisusing the vision vector and sensor vector for each region identified onthe vision matrix or sensor matrix as a particle. The control system 112may need to arbitrate when differing numbers of particles are identifiedbetween the vision and sensor matrices in the same corresponding areasof the matrices. For example, one matrix may indicate that there is oneparticle in a region and have an associated vector for that particlewhile the other matrix may indicate that there are two particles in thesame corresponding belt region with two associated vectors. The controlsystem 112 may arbitrate in this situation by summing, averaging, takingthe peak value, or otherwise combining the two associated vectors fromone matrix to provide values for comparison and use in the discriminantanalysis with the other matrix.

In one example, the control system 112 creates a first discriminant anda second discriminant as determined using a principle componentanalysis. Each discriminant may be a linear combination as a function ofthe parameters from the sensor and vision vectors. For example, a firstdiscriminant may be based on an equation such as aA+bB+cC+dD+ . . . ,where a, b, c, and d are constant values and A, B, C, and D areparameters from each of the vision and sensor vectors. A seconddiscriminant may be based on another equation such as mA+nB+oC+pD+ . . ., where m, n, o, and p are constant values and A, B, C, and D areparameters from each of the vision and sensor vectors. The first andsecond discriminants are used to separate and classify the particlesinto two or more classifications.

In one example, each of the discriminants are inputted as a pair into alookup table or chart by the control system 112, and if the pair ofdiscriminant is one of a series of predefined pairs, the particle isclassified into a predetermined category of material. In anotherexample, the control system 112 only uses a single discriminant asdetermined using a principle component analysis as described above, andthe control system 112 inputs the discriminant into a lookup table orchart, and/or compares the single discriminant to a cutoff threshold.FIG. 7 illustrates an example of a lookup table according to anembodiment, and is described in further detail below.

The functions or primary component analyses used in determining thediscriminants, the lookup tables and/or the cutoff thresholds may bedetermined during a calibration process using a known mixture ofmaterials. The functions, parameters and threshold may be chosen toprovide separation between cast and wrought materials with increasedpurity and recovery rates.

In further examples, additional discriminant analyses may besubsequently performed by the control system 112 using third and fourthdiscriminants, etc. to further refine the classification of the particleinto two categories, or to classify the particle into a third, fourth orother category of material.

At step 258, a diversion matrix is created by the control system 112.The diversion matrix is used by the control system 112 during thesorting and separating step 232 in FIG. 5. In other examples, thecontrol system 112 adds an additional array of diversion data to one ofthe image and sensor matrices for use in sorting and separating theparticles.

The diversion matrix may be based on the central frames (pre-stitching)of the vision and sensing matrices. In one example, the diversion matrixis sized based on the resolution or number of the ejectors 132, and maybe filled row by row, similar to a linescan image by the control system112. The diversion matrix may have a set number of rows that are filledand cycle based on a shift register, first-in-first-out algorithm.

The diversion matrix may be filled by the control system 112 using asorting value. The sorting value may be a binary value (0, 1) for abinary classification, or may be based on two-bit numbers, hexadecimal,or the like, for use in sorting of more than two classifications ofparticles 104.

In one example, the control system 112 assigns a sorting value based onthe classification as determined at steps 254 and 256 to all of thecells in the diversion matrix that correspond to the entirety of anidentified particle region, as determined using the vision pointerand/or the sensor pointer.

The combined vision and analog sensing system 100 provides significantimprovements in the purity and recovery in sorted scrap materials. Forexample, by using the vision system 106 with the analog sensing system108, sorting improvements are indicated for cast versus wroughtmaterials. In a conventional vision-only system with a single-pass,wrought materials had 85% purity and 45% recovery, and cast materialshad 49% purity and 88% recovery. In contrast, the system 100 accordingto the present disclosure operated with a single-pass and testing thesame feed materials at the same linear belt speed provided wroughtmaterials with 95% purity and 50% recovery, and cast materials with 56%purity and 96% recovery.

FIG. 9 is a plot of sample data for use is setting calibration andclassification parameters based on results obtained during experimentaltesting. FIG. 9 illustrates sample calibration data from the system 100that included cast and wrought aluminum materials. The wrought materialmay include sheet material. For each particle 104, the first and seconddiscriminants are plotted as pairs of data (PC1, PC2), with the firstdiscriminant (PC1) on the x-axis and second discriminant (PC2) on they-axis. As can be seen from the Figure, a cutoff threshold may beselected as line 270, such that materials on one side of the threshold270 are classified as cast and materials on the other side of thethreshold may be classified as wrought or other. In a further example,particles that are within a specified range of the threshold may beassigned an indeterminate classification to further increase purities.

FIG. 10 illustrates a flow chart for a method 300 of classifying aparticle 104 according to an embodiment and for use by the controlsystem 112 during step 230 of method 200 using the vision and sensorvectors as inputs. In other embodiments, various steps in the method 300may be combined, rearranged, or omitted. With reference to the method300, an example using mixed materials is used by way of a non-limitingexample, and one of ordinary skill in the art would understand how toapply the method 300 to classifying other groupings of materials. Themixed materials include a mixture of “bare” metals, including painted orcoated metals, and metals that are embedded or encapsulated in non-metalmaterials. In one example, the mixed materials include metal wire thatis coated with a layer of insulation, portions of circuit boards andother electronic waste, scrap materials from tire with metal beltembedded in rubber, and other metals that are at least partiallyentrapped, encapsulated, or embedded within insulation, rubber,plastics, or other nonconductive materials. The mixed materials 104 maybe provided as nonferrous materials that contain other metal and metalalloys. The particles 104 may be wetted prior to passing through thevision and sensing systems 106, 108, or may be provided as dry particlesthrough the system 100. The vision vector and sensor vector are used asinputs to a lookup table conducted by the control system 112implementing method 300 to classify the scrap particles 104 for a binarysort, tertiary sort, etc.

At step 302, the control unit inputs the vision and sensor vectors. Atstep 304, the control unit 112 conducts an initial thresholdclassification using the vision vector. The vision vector may containdata such as [R, G, B] values for each pixel in a region identified ascontaining a particle 104, as well as the vision pointer, and othershape features, such as a width factor. The width factor may be anaverage minimum width for the particle 104, or another value associatedwith width. The control unit 112 may use various techniques to determineat step 304 if the particle falls within a first classification ofmaterials, such as a circuit board.

For example, the control unit 112 may classify the particle using ahistogram analysis. In another example, the control unit 112 uses amulti-discriminant analysis to reduce the three (or more) colorcomponents to two color components as a pair of discriminants. Thecontrol unit 112 then inputs the pair of discriminants for each pixel orcell associated with the particle into a calibration table or chartstored in a memory unit associated with the control unit. An example ofa calibration table is illustrated in FIG. 11, and may be non-linear asshown by region 320. If the pair of discriminants is one of a series ofpredefined pairs of discriminants and falls within region 320, asdetermined via a calibration process, the control unit 112 flags thepixel or cell in the image, for example, with a 1 value. If the pair ofdiscriminants is not one of the series of predefined pairs ofdiscriminants such that it falls outside of region 320, the control unit112 leaves the pixel or cell in the image unflagged, for example, as a 0value. The control unit 112 calculates a color input for the particle104 based on the number of flagged pixels or cells, for example, bydetermining a fill fraction for the particle 104 by normalizing ordividing the summation of the flagged pixels by the total number ofpixels associated with the particle. The control unit may then conductan initial classification by comparing the color input or fill fractionto a threshold value. For example, the control unit 112 may classify ascrap particle as being circuit board during this step.

The control unit 112 may provide classification data to a diversionmatrix at block 306, with the diversion matrix structured similarly tothat described above with respect to FIG. 6.

At step 308, the control unit 112 conducts another initialclassification step using data from both the vision and sensor vectors,and continues to populate the diversion matrix as appropriate. Thecontrol unit 112 continues to analyze the vision and sensor vectors tofurther classify mixed material particles on the belt 104 that were notidentified and classified at step 304. For example, for particlesidentified only in the vision vector and being without a correspondingidentified region in the sensor vector, the control unit may fill thediversion matrix at block 306 with a classification of non-metalmaterials.

At step 310, the control unit 112 classifies the particles on the beltidentified in both the vision and sensor vectors, and populates thediversion matrix accordingly. In one example, the control unit 112implements a decision tree classification technique. The control unit112 may use value as determined from the sensor and vision vectors incomparison with various threshold values to classify the particle intoone or more categories of material. An example of a decision tree foruse in step 310 is illustrated in the Figure, and the decision tree maybe changed based on the associated classifications of materials andmixture of scrap materials for sorting. The control unit 112 compares afirst value, w, from the vision vector, such as a shape or width factor,to a threshold value, A. If the value is less than the threshold A, thecontrol unit 112 then compares a second value, v, from the sensorvector, such as a peak voltage or average voltage, to a threshold valueB, and then classifies the particle into a first or second category ofmaterial, such as metal or wire, respectively. If the value is greaterthan the threshold A, the control unit 112 then compares the secondvalue, v, from the sensor vector, such as a peak voltage or averagevoltage, to a threshold value C, and then classifies the particle into afirst or second category of material, such as metal or wire, or intoanother category of material. Of course, in other examples, other valuesmay be used from the vision and sensor vectors, more than two values maybe used in the decision tree, and the decision tree may be structured inanother manner.

While exemplary embodiments are described above, it is not intended thatthese embodiments describe all possible forms of the disclosure. Rather,the words used in the specification are words of description rather thanlimitation, and it is understood that various changes may be madewithout departing from the spirit and scope of the disclosure.Additionally, the features of various implementing embodiments may becombined to form further embodiments of the disclosure.

What is claimed is:
 1. A method of sorting scrap particles comprising:imaging a moving conveyor containing scrap particles using a visionsystem to create a vision image corresponding to a timed location of theconveyor; employing a control system to analyze the vision image as avision matrix of cells, identify cells in the vision matrix containing aparticle, and generate a vision vector containing vision data from thevision matrix for the particle; sensing scrap particles contained on themoving conveyor using a sensing system to create a sensing matrixcorresponding to the timed location of the conveyor, the sensing systemhaving at least one array of analog proximity sensors; employing thecontrol system to analyze the sensing matrix, identify cells in thesensing matrix containing a particle, and generate a sensing data vectorcontaining sensing data from the sensing matrix for the particle; andemploying the control system to classify the particle into one of atleast two classifications of a material as a function of the vision datavector and the sensing data vector by inputting the vectors into amachine learning algorithm.
 2. The method of claim 1 wherein the controlsystem is employed to stitch the vision image with a preceding visionimage and a subsequent vision image prior to analyzing the vision image.3. The method of claim 1 wherein the control system is employed tostitch the sensing matrix with a preceding sensing matrix and asubsequent sensing matrix prior to analyzing the sensing matrix.
 4. Themethod of claim 1 further comprising employing the control system toquantize an analog signal from each of the sensors in the at least onearray while maintaining a predetermined resolution from the analogsignal; and wherein each sensor in the array of sensors is positioned ata first distance away from the conveyor.
 5. The method of claim 1wherein the vision data vector contains a vision pointer and at leastone of the following: at least one color input, a busyness score, aperimeter value, at least one hole determination, and an area ratio. 6.The method of claim 1 further comprising employing the control system tocalculate another value for one of the vision and sensor vectors, theanother value calculated from vision and sensor data for the sameparticle.
 7. The method of claim 1 wherein the control system is furtheremployed to arbitrate the sensor and vision vectors in response to thesensor and vision matrices indicating a differing number of particlesover the same corresponding area between the matrices.
 8. The method ofclaim 1 wherein the machine learning algorithm comprises at least one ofa support vector machine, a partial least squares discriminant analysis,a neural network, a partial least squares discriminant analysis, arandom forest of decision trees.
 9. The method of claim 1 wherein thecontrol system is employed to identify and analyze particles in thevision image using a location as determined from the sensing matrix. 10.The method of claim 1 further comprising sorting the particle bycontrolling a separator device based on the classification for theparticle and the timed location of the conveyor.
 11. A method of sortingscrap particles comprising: imaging a moving conveyor containing scrapparticles using a vision system to create a vision image correspondingto a timed location of the conveyor; employing a control system toanalyze the vision image as a vision matrix of cells, identify cells inthe vision matrix containing a particle, and generate a vision vectorcontaining vision data from the vision matrix for the particle; sensingscrap particles contained on the moving conveyor using a sensing systemto create a sensing matrix corresponding to the timed location of theconveyor, the sensing system having at least one array of analogproximity sensors; employing the control system to analyze the sensingmatrix, identify cells in the sensing matrix containing a particle, andgenerate a sensing data vector containing sensing data from the sensingmatrix for the particle; and employing the control system to classifythe particle into one of at least two classifications of a material as afunction of the vision data vector and the sensing data vector; whereinthe sensing data vector contains a sensor pointer and at least one ofthe following: a peak voltage, a rate of change of voltage, an averagevoltage, a summation of the voltages over a particle area, an area ratiofactor, and a compactness factor.
 12. The method of claim 11 wherein thecontrol system classifies the particle as the function of the visiondata vector and the sensing data vector using a principal componentanalysis with at least one discriminant as a function of the vision dataand sensor data.
 13. The method of claim 11 wherein the control systemclassifies the particle as the function of the vision data vector andthe sensing data vector using a decision tree and at least one valuefrom each of the vision data vector and the sensing data vector.
 14. Themethod of claim 11 further comprising calculating a color input for theparticle from a color model by determining color components for eachcell in the vision matrix associated with the particle, the color inputcalculated by inputting two color components from each cell of thematrix associated with the particle as a pair of discriminants into acalibration table, flagging the cell if the pair of discriminants is oneof a predefined pair of discriminants, and calculating the color inputby normalizing a summation of the flagged cells by a total number ofcells associated with the particle.
 15. The method of claim 14 furthercomprising employing the control system to pre-classify the particleusing the color input prior to classifying the particle as the functionof the vision data vector and the sensing data vector.
 16. A system forsorting randomly positioned scrap material particles on a movingconveyor, the system comprising: a vision system having an imagingsensor and an illuminated predefined viewing area to image a conveyorpassing therethrough at a time interval; a sensing system having anarray of analog inductive proximity sensors arranged in a single commonplane arranged generally parallel to the conveyor and positioned tosense scrap particles on the conveyor; and a control system configuredto receive and process image data acquired from the vision system toidentify a scrap particle on the conveyor in the viewing area, analyzethe vision data for the particle to form a vision data vector, receiveand process sensor data acquired from the sensing system and timed tocorrespond with the vision data to identify the scrap particle on theconveyor, analyze the sensor data for the particle to form a sensor datavector, and classify the particle into a classification of materialusing the vision data vector and the sensing data vector; wherein thevision data vector comprises a vision pointer, and at least one of thefollowing: at least one color input, a busyness score, a perimetervalue, at least one hole determination, and an area ratio; and whereinthe sensor data vector comprises a sensor pointer and at least one ofthe following: a peak voltage, a rate of change of voltage, an averagevoltage, a summation of the voltages over a particle area, an area ratiofactor, and a compactness factor.
 17. The system of claim 16 furthercomprising a sorting system configured to sort the particle on theconveyor in response to receiving the classification from the controlsystem.
 18. The system of claim 16 wherein the vision system ispositioned subsequent to the sensing system along the conveyor.
 19. Amethod of sorting scrap particles comprising: imaging a moving conveyorcontaining scrap particles using a vision system to create a visionimage corresponding to a timed location of the conveyor; employing acontrol system to analyze the vision image as a vision matrix of cells,identify cells in the vision matrix containing a particle, and generatea vision vector containing vision data from the vision matrix for theparticle; sensing scrap particles contained on the moving conveyor usinga sensing system to create a sensing matrix corresponding to the timedlocation of the conveyor, the sensing system having at least one arrayof analog proximity sensors; employing the control system to analyze thesensing matrix, identify cells in the sensing matrix containing aparticle, and generate a sensing data vector containing sensing datafrom the sensing matrix for the particle; employing the control systemto classify the particle into one of at least two classifications of amaterial as a function of the vision data vector and the sensing datavector; and employing the control system to arbitrate the sensor andvision vectors in response to the sensor and vision matrices indicatinga differing number of particles over the same corresponding area betweenthe matrices.
 20. A method of sorting scrap particles comprising:imaging a moving conveyor containing scrap particles using a visionsystem to create a vision image corresponding to a timed location of theconveyor; employing a control system to analyze the vision image as avision matrix of cells, identify cells in the vision matrix containing aparticle, and generate a vision vector containing vision data from thevision matrix for the particle; sensing scrap particles contained on themoving conveyor using a sensing system to create a sensing matrixcorresponding to the timed location of the conveyor, the sensing systemhaving at least one array of analog proximity sensors; employing thecontrol system to analyze the sensing matrix, identify cells in thesensing matrix containing a particle, and generate a sensing data vectorcontaining sensing data from the sensing matrix for the particle; andemploying the control system to classify the particle into one of atleast two classifications of a material as a function of the vision datavector and the sensing data vector; wherein the control system isemployed to identify and analyze particles in the vision image using alocation as determined from the sensing matrix.